An auto-scaling framework for analyzing big data in the cloud environment

Processing big data on traditional computing infrastructure is a challenge as the volume of data is large and thus high computational complexity. Recently, Apache Hadoop has emerged as a distributed computing infrastructure to deal with big data. Adopting Hadoop to dynamically adjust its computing resources based on real-time workload is itself a demanding task, thus conventionally a pre-configuration with adequate resources to compute the peak data load is set up. However, this may cause a considerable wastage of computing resources when the usage levels are much lower than the preset load. In consideration of this, this paper investigates an auto-scaling framework on cloud environment aiming to minimise the cost of resource use by automatically adjusting the virtual nodes depending on the real-time data load. A cost-effective auto-scaling (CEAS) framework is first proposed for an Amazon Web Services (AWS) Cloud environment. The proposed CEAS framework allows us to scale the computing resources of Hadoop cluster so as to either reduce the computing resource use when the workload is low or scale-up the computing resources to speed up the data processing and analysis within an adequate time. To validate the effectiveness of the proposed framework, a case study with real-time sentiment analysis on the universities’ tweets is provided to analyse the reviews/tweets of the people posted on social media. Such a dynamic scaling method offers a reference to improving the Twitter data analysis in a more cost-effective and flexible way

Patient preferences for whole-body MRI or conventional staging pathways in lung and colorectal cancer: a discrete choice experiment

Objectives To determine the importance placed by patients on attributes associated with whole-body MRI (WB-MRI) and standard cancer staging pathways and ascertain drivers of preference. Methods Patients recruited to two multi-centre diagnostic accuracy trials comparing WB-MRI with standard staging pathways in lung and colorectal cancer were invited to complete a discrete choice experiment (DCE), choosing between a series of alternate pathways in which 6 attributes (accuracy, time to diagnosis, scan duration, whole-body enclosure, radiation exposure, total scan number) were varied systematically. Data were analysed using a conditional logit regression model and marginal rates of substitution computed. The relative importance of each attribute and probabilities of choosing WB-MRI-based pathways were estimated. Results A total of 138 patients (mean age 65, 61% male, lung n = 72, colorectal n = 66) participated (May 2015 to September 2016). Lung cancer patients valued time to diagnosis most highly, followed by accuracy, radiation exposure, number of scans, and time in the scanner. Colorectal cancer patients valued accuracy most highly, followed by time to diagnosis, radiation exposure, and number of scans. Patients were willing to wait 0.29 (lung) and 0.45 (colorectal) weeks for a 1% increase in pathway accuracy. Patients preferred WB-MRI-based pathways (probability 0.64 [lung], 0.66 [colorectal]) if they were equivalent in accuracy, total scan number, and time to diagnosis compared with a standard staging pathway. Conclusions Staging pathways based on first-line WB-MRI are preferred by the majority of patients if they at least match standard pathways for diagnostic accuracy, time to diagnosis, and total scan number